Learning Deployable Locomotion Control via Differentiable Simulation

C. Schwarke, V. Klemm, J. Bagajo, J.P. Sleiman, I. Georgiev, J. Tordesillas Torres, M. Hutter

Conference on Robot Learning - CoRL 2025, Seoul (South Korea). 27-30 September 2025

Summary:

Differentiable simulators promise to improve sample efficiency in robot learning by providing analytic gradients of the system dynamics. Yet, their application to contact-rich tasks like locomotion is complicated by the inherently nonsmooth nature of contact, impeding effective gradient-based optimization. Existing works thus often rely on soft contact models that provide smooth gradients but lack physical accuracy, constraining results to simulation. To address this limitation, we propose a differentiable contact model designed to provide informative gradients while maintaining high physical fidelity. We demonstrate the efficacy of our approach by training a quadrupedal locomotion policy within our differentiable simulator leveraging analytic gradients and successfully transferring the learned policy zero-shot to the real world. To the best of our knowledge, this represents the first successful sim-to-real transfer of a legged locomotion policy learned entirely within a differentiable simulator, establishing the feasibility of using differentiable simulation for real-world locomotion control.

Spanish layman's summary:

El artículo presenta un modelo de contacto diferenciable que permite entrenar una política de locomoción cuadrúpeda en simulación y transferirla zero-shot a un robot real, logrando una transferencia sim-to-real exitosa mediante un simulador totalmente diferenciable.

English layman's summary:

The paper presents a differentiable contact model that enables training a quadruped locomotion policy in simulation and transferring it zero-shot to a real robot, leading to a successful sim-to-real transfer using a fully differentiable simulator.

Keywords: Differentiable Simulation, Contact Modeling, Quadruped Locomotion

Published in: Proceedings of Machine Learning Research, vol. 305, pp: 3665-3684

Publication date: 30-Sep-2025.

Citation:
C. Schwarke, V. Klemm, J. Bagajo, J.P. Sleiman, I. Georgiev, J. Tordesillas Torres, M. Hutter, "Learning Deployable Locomotion Control via Differentiable Simulation", presented at Conference on Robot Learning - CoRL 2025, Seoul, South Korea, 27-30 September 2025. In: Proceedings of Machine Learning Research, vol. 305, pp. 3665-3684

IIT-25-255C